Sound Sculpting: Manipulating Sound through Virtual Sculpting

We have prototyped sound sculpting, an environment in which a sound artist or musician can create, edit or perform sounds by changing parameters, like position, orientation and shape of a virtual object as input device, that can only be perceived through its visual and acoustic representations. We implemented sound sculpting by extending a realtime, visual programming language called Max/FTS, which was originally designed for sound synthesis, with software objects to interface datagloves and 6DOF sensors and to compute human movement and virtual object features. To date, we have performed two pilot studies, involving a virtual input device with a behaviour of a rubber balloon and another with a behaviour of a rubber sheet for the control of sound spatialization and timbre parameters. It was found that the more physical analogies are used for a mapping the faster the mapping can be learned. This requires more research for mapping virtual object features to timbre naturally. While both hands can be used for manipulation, left-hand-only sound sculpting may be a useful replacement for and extension of the standard keyboard modulation wheel. While we have succesfully implemented and tested virtual sculpting as a new real-time interaction method for sound design and musical performance, we need to address more manipulation pragmatics in the mapping and more carefully identify movement features to map to virtual object parameters.